Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Split googleapis into per-language modules #3472

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

Split googleapis into per-language modules #3472

wants to merge 11 commits into from

Conversation

fmeum
Copy link
Contributor

@fmeum fmeum commented Dec 21, 2024 •
edited
Loading

Introduces new googleapis-* modules that contain the per-language module dependencies needed to support the individual rules used by the main googleapis module. This allows BCR modules to declare dependencies on language-specific rules in googleapis without requiring that module to depend on every possible language ruleset, which isn't sustainable.

This results in a breaking change for root modules that previously used the switched_rules extensions. These modules now need to explicitly declare deps on the googleapis-* flavors they need, but will be guided through the process by error messages.

In the future, new flavors can be added in a backwards compatible way.

googleapis-grpc-cc is blocked on #3470, which adds a release of grpc including grpc/grpc@3f001f7.

@bazel-io
Copy link
Member

Hello @bazelbuild/bcr-maintainers, modules without existing maintainers (googleapis-cc, googleapis-go, googleapis-java, googleapis-python, googleapis) have been updated in this PR. Please review the changes.

@fmeum fmeum force-pushed the fmeum/googleapis-0.0.0-20241220-5e258e33 branch 2 times, most recently from dfcb309 to 0d8947f Compare December 21, 2024 12:49
@fmeum fmeum added presubmit-auto-run Presubmit jobs will be triggered for new changes automatically without reviewer's approval skip-url-stability-check Skip the URL stability check for the PR skip-source-repo-check In BCR presubmit, skip checking the source repository for the module added in the PR labels Dec 21, 2024
@fmeum fmeum force-pushed the fmeum/googleapis-0.0.0-20241220-5e258e33 branch 2 times, most recently from 8f7654a to c54877c Compare December 21, 2024 13:37
@bazel-io
Copy link
Member

Hello @bazelbuild/bcr-maintainers, modules without existing maintainers (googleapis-cc, googleapis-go, googleapis-grpc-cc, googleapis-grpc-java, googleapis-java, googleapis-python, googleapis) have been updated in this PR. Please review the changes.

@fmeum fmeum force-pushed the fmeum/googleapis-0.0.0-20241220-5e258e33 branch from 4d80faa to 979ea95 Compare December 21, 2024 14:25
@bazel-io
Copy link
Member

Hello @bazelbuild/bcr-maintainers, modules without existing maintainers (googleapis-cc, googleapis-go, googleapis-grpc-java, googleapis-java, googleapis-python, googleapis) have been updated in this PR. Please review the changes.

@fmeum fmeum force-pushed the fmeum/googleapis-0.0.0-20241220-5e258e33 branch from 60cfbd8 to 6617794 Compare December 21, 2024 15:03
fmeum
fmeum commented Dec 21, 2024
View reviewed changes
modules/googleapis-cc/0.1.0/source.json
@@ -0,0 +1,7 @@
{
"integrity": "sha256-hznHbmgfkAkjuQDJ3w73XPQh05yrtUZQxLmtGbanbYU=",
"url": "https://github.com/fmeum/bazel-central-registry/releases/download/v1.0.0/empty.zip",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Replace with a better source after #3474.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't we just allow url to be not set when overlay is used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could, but that would require raising the minimum Bazel version to at least 8.1.0 (the earliest we can ship this).

@fmeum fmeum changed the title Add googleapis 0.0.0-20241220-5e258e33 Split googleapis into different modules Dec 21, 2024
@fmeum fmeum marked this pull request as ready for review December 21, 2024 16:00
@fmeum fmeum mentioned this pull request Dec 21, 2024
Closed
@fmeum fmeum changed the title Split googleapis into different modules Split googleapis into per-language modules Dec 21, 2024
@fmeum fmeum force-pushed the fmeum/googleapis-0.0.0-20241220-5e258e33 branch from 0d8efa5 to de336d6 Compare December 25, 2024 20:42
@fmeum fmeum force-pushed the fmeum/googleapis-0.0.0-20241220-5e258e33 branch 2 times, most recently from 2ad8da5 to 432a13e Compare December 25, 2024 20:50
@fmeum fmeum marked this pull request as draft December 25, 2024 21:31
@mering
Copy link
Contributor

mering commented Dec 26, 2024

Another thing which just came to my mind: Uploading new versions will become much more work with this approach. This is especially relevant since we don't have active maintainers for this package and are even further away from release automation. So it will become less likely that people contribute version updates.

@fmeum
Copy link
Contributor Author

fmeum commented Dec 27, 2024

We should avoid making updates more complicated, but I don't see why this would make it so. Instead of the patches, there are now a few overlay files, but that's the only change I see. The "flavor" modules only need new versions if the targets generated in googleapis change in a fundamental way. Do you see other downsides to this approach?

I'm thinking of adding a tool to the repo that creates a new version of an existing module by symlinking over files that haven't changed and replacing the old version with the new version in module files, URLs and prefixes. That should make it very easy to update googleapis, regardless of which approach is used. What do you think about that?

@fmeum fmeum force-pushed the fmeum/googleapis-0.0.0-20241220-5e258e33 branch from 432a13e to d7bf807 Compare December 30, 2024 08:35
@fmeum fmeum force-pushed the fmeum/googleapis-0.0.0-20241220-5e258e33 branch from d7bf807 to 0398bd8 Compare December 30, 2024 08:39
@fmeum fmeum marked this pull request as ready for review December 30, 2024 09:00
@mering
Copy link
Contributor

mering commented Dec 30, 2024 •
edited
Loading

We should avoid making updates more complicated, but I don't see why this would make it so. Instead of the patches, there are now a few overlay files, but that's the only change I see. The "flavor" modules only need new versions if the targets generated in googleapis change in a fundamental way. Do you see other downsides to this approach?

I raised everything which was on my mind for now in #3437 and here.

I'm thinking of adding a tool to the repo that creates a new version of an existing module by symlinking over files that haven't changed and replacing the old version with the new version in module files, URLs and prefixes. That should make it very easy to update googleapis, regardless of which approach is used. What do you think about that?

It would be great if such tool would also handle patches (at least in cases where it applies with somewhat larger fuzzing for moved lines and skip already applied patches). It could create a temporary git repo for this for example (git init) and delete the .git folder on cleanup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mering mering mering left review comments

@Wyverald Wyverald Awaiting requested review from Wyverald

@meteorcloudy meteorcloudy Awaiting requested review from meteorcloudy

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
presubmit-auto-run Presubmit jobs will be triggered for new changes automatically without reviewer's approval skip-source-repo-check In BCR presubmit, skip checking the source repository for the module added in the PR skip-url-stability-check Skip the URL stability check for the PR
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@fmeum @bazel-io @mering